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Abstract 

We consider the recursive estimation of a regression functional where the explanatory variables 



fvj , take values in some functional space. We prove the almost sure convergence of such estimates for 

, ■ dependent functional data. Also we derive the mean quadratic error of the considered class of 

^ i' estimators. Our results are established with rates and asymptotic appear bounds, under strong 

■^^ • mixing condition. 
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1 Introduction 



In this paper we study the regression model of a scalar response variable given a functional covariate. 
Functional data analysis is a problem of considerable interest in statistics and has been found to 
be useful in many practical fields, including climatology, economics, linguistics, medicine,... The 

f^ I statistical study of this kind of data is the subject of many papers in parametric and nonparametric 
statistics. For background material on this subject we highlight the works of Ramsay and Dalzell 
|17j . Ramsay and Silverman |181 I19j . Since these pioneer contributions, the literature on this topic 
is still growing. A survey of the nonparametric functional regression appears in Ferraty et al. [llj . 

f^ , while more recent results are collected in the book by Ferraty and Vieu [lOl- There are several ways 

cn I to study the link between a response variable given an explanatory variable. For example, one of the 
most studied models is the regression model when the response variable Y is real and the explanatory 
variable X belongs to some functional space £. Then, the regression model writes Y = r{X) + e, 

S^ I where r : iS — )• M is an operator and e is an error random variable. Many works have been done 
H ■ around this model when the operator r is supposed to be linear, contributing to the popularity of 

■ ■ ■ the so-called functional linear model. We refer the reader for instance to the works of Cardot et al. 
[1] or Crambes et al. [5j for different methods to estimate r in this linear context. Another way 
is to estimate r by a nonparametric approach. The first results on this context were obtained by 
Ferraty and Vieu [8] . They established the almost complete convergence of a kernel estimator of the 
regression function in the i.i.d case. The study on their Nadaraya- Watson type estimator is extended 
to several directions. Dabo-Niang and Rhomari [6j stated the L^-convergence of the kernel estimator, 
while Delsol [^ gave the L^-convergence with asymptotic appear bound. The asymptotic normality of 
the same estimator has been obtained by Masry [15j under strong mixing conditions and extended by 
Delsol [7]. Ling and Wu [14] stated the almost sure convergence of the kernel estimator under strong 
mixing conditions. Functional data appear in many practical situations, as soon as one is interested 
on a continuous phenomenon. To consider such data as objects belonging to some functional space 



*EQUIPPE 

^EQUIPPE, Universite Charles De Gaulle, Lille 3, Maison de la Recherche, Domaine universitaire du Pont de Bois, 
BP 60149, 59653 Villeneuve d'ascq cedex, France, baba.thiani@univ-lille3.fr 



brings more precisions on the studied phenomenon. However, the computation of the estimators can 
be time consuming in this context, the use of recursive methods remains a good alternative to the 
classical ones. By 'recursive', we mean that the estimator calculated from the first n observations, say 
fn, is only a function of fn-i and the n^^ observation. In this way, the estimator can be updated with 
each new observation added to the database. 

The purpose of this paper is to apply recursive methods to functional data. Recursive estimation is 
achieved with the use of recursive estimators, typically kernel ones. For informations on nonparametric 
recursive methods, the reader is referred to the books by Gyorfi et al. [13], or the recent works of 
Vilar and Vilar |21j . Wang and Liang [22], Quintela-Del-Rio [16], Amiri [1] and the references there in. 
The first results concerning the recursive kernel estimator of the regression function with functional 
explanatory variable were obtained by Amiri et al. [2]. They established the mean square error, the 
almost sure convergence with rates and a central limit theorem for a class of recursive kernel estimates 
of the regression function when the explanatory variable is functional and the observations are i.i.d. 
The main goal of this paper is the extension of a few of the results obtained by Amiri et al. [2j to 
dependent data. The rest of the paper proceeds as follows. We will present the regression model on 
section [2J On section O we give assumptions and results on the strong consistency and mean quadratic 
error for the recursive regression estimate. Section 2] is devoted to the proofs of our results. 

2 Recursive regression estimate for curves 

Let us consider a random process Zt = {Xt,Yt),t £ N, where Yj is a scalar random variable and Xt 
takes values in some functional space £ endowed with a semi- norm || • ||. Assume the existence of an 
operator r satisfying r(x) := E(yj|A'j = x) ) X ^ "S", for all t G N. To estimate r, one can consider 
the family of recursive estimators indexed by a parameter i G [0, 1] introduced in Amiri et al. and 
defined by 



rlfl (X) := 



E 



Y, 



Fihi)- 



^K 
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'K 



\X-X^\ 

hi 



where iC is a kernel, {hn) a sequence of bandwidths and F is the cumulative distribution function of the 
random variable ||x— A"!! . This family of estimators is a recursive modification of the Nadaraya- Watson 
type estimator of Ferraty and Vieu [TO] and can be computed recursively by 



' n+ 



i(x) 



n 

i=l 



l-£ 



'^f{x) + 



'n+l 

E nh^) 

.1=1 



l-t 



y„+i4'ii (llx 



rC-i 



n+l\ 



E nh^Y- 

i=l 



/^'(X) + 



n+l 

E^(^. 

.i=l 



,1-f 



K. 



n+l 



IX 



X, 



n+l I 



with 



E 



Y,. T^ \\X-X,\\ 
F{h,Y^ V ^» 



E nh^Y- 



"j In vXj 



E 
j=i 



F(h,} 



^K 



\x-XA 

hi 



(1) 



E nh^Y- 

i=l 



and Kf\-) : 



K [j^]. The recursive property of this class of regression estimators 



j=i 



offers many advantages and is clearly useful in sequential investigations and also for a large sample 
size. Indeed, this kind of estimators are of easy implementation and interpretation, fast to compute 
and they do not require extensive storage of data. The weak and strong consistency of this family of 
estimators was studied by Amiri et al. [2] in the framework of the independent case. 



3 Assumptions and main results 

3.1 Assumptions 

In the same spirit as Masry [15], we suppose throughout the paper the existence of nonnegative func- 
tions /i and (f) such that (j){Q) = and F{h) = P [||x — ^^H <h]= 4'{h)fi{x), for /i on a neighborhood 
of zero. Then (p is an increasing function of h and (/"(/i) — )• as /i — )• 0. The function /i is referred to 
as a functional probabihty density (see Gasser et al. [12] for more details). We will assume that the 
following assumptions hold. 

(HI) The operators r and a'^ are continuous on a neighborhood of x- Moreover, the function 
({t) := E [{r(A:') — r{x)} / \\X — x\\ = A is assumed to be derivable at t = 0. 

(H2) K is nonnegative bounded kernel with support on the compact [0, 1] such that inf K{t) > 0. 

te[o,i] 

(H3) For any s G [0, l],Th{s) := ^ ^ Ms) < oo as /i ^ 0. 



4>{h) 



(H4) (i) hn i 0, n(t){hn) -> oo, A. 
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4>{hi"^ 
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(ii) Vr < 2, Bn,r ■=-y\ 



n . 
1=1 



4>{hn) 



<P{hn) 

/3[r] > 0, as n — ;■ oo. 



Q[£] > as n — 7- oo. 



(Inn) '' 
(m) For any /U > 0, lim — - — — = 0. 

n-i>oo n<p{nn) 

(H5) (i) {Xt)teN is a strong mixing process with ax{k) < ck~P, k > 1, for some c > and p > 2. 

Hh) 



(ii) There exist non negative functions il) and /2 such that ip{h) — t- as /i — t- 0, the ratio 



is bounded and supF[{Xi,Xj) G B{x,hi) x B{x,hj)] < ip{hi)tlj{hj)f2{x)- 

(H6) There exist A > and ^u > such that E [exp (A|y|^)] < oo. 

Since this paper is a generalization to dependent case of the results in Amiri et al. ^, several of 
the assumptions are the same as those used in the earlier reference. The reader is then referred to this 
last for more comments on assumptions. Let us mention that the decrease of the sequence (/in)nGN 
is particular to the recursive estimators and for dependent data. The technical condition (H4)(iii) is 
unrestrictive and is easily satisfied by the popular choices of and h^ given by <\){hri) ~ n~^, with 
< ^ < 1. Assumption (H5)(i) is the classically strong mixing condition which, is well known to be 
satisfied by linear or stationary ARMA processes. In order to simplify the presentation, we assume 
the strong mixing coefficient to be arithmetic, but the main results can be obtained under several 
conditions on this coefficient. Assumption (H5)(ii) plays a crucial role in our calculus, when we show 
the negligibility of some covariance terms. It has been used by Masry [15] in the non recursive case. 
Finally, as developped in Amiri et al. [2J, assumption (H6) implies that 

E ( max \Yi\A. = 0[{lnnf/''],\fp > l,n > 2. (2) 

yl<i<n J 

3.2 Main results 

For convenience, let us introduce the following notations: 

Mo = K{1) - I {sK{s))'To{s)ds, Ml = K{1) - [ K'{s)To{s)ds 
Jo Jo 

M2 = K\l) - t\K\s))'To{s)ds. 
Jo 



In the following theorem, we establish the almost sure convergence of the proposed recursive kernel 
estimator of the regression function. 

Theorem 1 Assume that (H1)-(H6) hold. If lim nh^ = 0, then 



lim sup 

n— ^-oo 



n(p{hn) 
In n 



1/2 



riHx)-r{x) <j^^[l + Ve{x)] a.s. 



where 
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Mo 



(3) 



for all X such that fi{x) > 0- 

Theorem [T] is an extension of Ferraty and Vieu's [5] result on functional kernel-type estimate to 
the general family of recursive estimators rii (x)- A similar result is also obtained by Ling and Wu 
|14j for a truncated version of the Nadaraya- Watson type estimator, under the condition E|y| < oo, 
which is weaker than assumption (H6). However, Theorem [1] establishes the rate of convergence with 
exact appear bound, while Ling and Wu's [14j result tells only the rate of convergence in function of 
the variances of the numerator and denominator of the estimator. As we will see in the proofs below, 
assumption (H6) will be necessary, for the study of the covariance terms and also when we shall prove 
the cancellation of the residual term between the estimator and its truncated version. Finally, let 
us mention that compared with the result in Amiri et al. [2], as in the multivariate framework, it is 

difficult to obtain the optimal rate ? in n ^^ ^^^ dependent case. 



The mean square error of r„ (x) is given in Theorem [2] below. 
Theorem 2 Under assumptions (H1)-(H6), 
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[1 + 0(1) 



for all X such that fi{x) > 0- 



Theorem [2] is an extension to functional data of the result of Amiri [1 ' in finite dimensional setting. 
Also, our result generalizes the works of Bosq and Cheze-Payaud |3j to functional and recursive setting. 
Finally, in counterpart of the almost sure convergence. Theorem [2] gives the same rate of convergence 
and asymptotic constants as those obtained for the iid case in Amiri et al. [2]. 

4 Proofs 



In the sequel, and through the paper, c will denote a constant whose value is unimportant and may 
vary from line to line. Also, we set 

'\x-x,\' 



Kiix) = K 



hi 



Finally, for convenience we will use the following decomposition 

rlfl(x) - r{x) = '^"'(^)-:(^)^"^(^) + '^"^(^) - '^"' W 



/J^^(X) 
where (pn (x) is a truncated version of ipn {x) defined by 



I'n\x) 



^L^kx) 
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1=1 
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\X-Xi\ 



^^ FihiY"^^''^ -"'-'" \ hi 
bn being a sequence of real numbers which goes to +oo as n — )• oo. 



(4) 



(5) 



4.1 Preliminary lemmas 

In order to prove the main results, we need the following lemmas. 

Lemma 1 Under assumptions (H1)-(H4), we have 

E 



E 



^ifkx) 



fl^\x) 



P[i-i] ^h 



Proof. See Amiri et al. [2], since the bias term is not depending to the mixing structure. 
Lemma 2 Under assumptions (H1)-(H6), we have 

/?[l-2£] Ms 1 
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Var[^^{x) 

' f]^\x),^^{x) 



^fi_^] hix) n(t){hn) 



[! + «(!)] 
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P[i-e.] 



2, ,. Ms [l + o(l) 



fi{x) n(l){hn) 



Ai-2^] f . Ma [l + o(l)] 



/or a// X such that /i(x) > 0. 
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Proof. The variance term of fhix) ^^^ be decomposed in variance and covariance terms as 



Var(/M(x)) 



E^(^^)'" 



j=i 



-2 






(6) 



1=1 



where for any integers i and j, Ajj = F[hi) ^F{hj) ^ Coy {Ki{x),Kj{x)) ■ Noting that the principal 
term Fi in the right-hand side of ^ corresponds to the variance term of fn in the independent case 
(see Amiri et al. [2] fore more details), and is given by 

n<^(/.„)Fi = §^-^ [1 + 0(1)]. 

Now, let us establish that the covariance term F2 is negligible. To this end, let c„ be a sequence of 
real numbers tending to 00 as n — )• 00. We can write 



(7) 



Fo ^ Fj :^ Fo-\ -\- Foo. 

Prom assumptions (H2) and (H5)(ii), we have for any i ^ j 



nK:{x)K, 



i(x)] = / 

JfO, 



K(u)K(v 



\X-Xi\\ \\X-Xj\ 



[0,1] X [0,1] 



{u,v) 



< \\K\ 



\X-Xi\\ < hi,\\x- XjW < hj 



< c'4){hi)ip{hj). 
Note that, from the proof of Lemma 2 in Amiri et al. [2] we can write 



(8) 



E[K,{x)] = Hh^)f lix) 



Kil)- f K'{s)ThM)ds 

JO 



so, we get 



\Cov{Ki{x),K,{x))\ < c[^{hi)i:{hj)+(t>{hi)<l){hj)]. 



(9) 



Hence, we deduce that 



■>P{hk+p)->P(hp) 



l-(-A(h \l-t 



F2I < 



cElu T.U mSm^ + ^(^^+p)^-^'^(^.) 



:— -^211 + F212- 



Now, Assumption (H5)(ii) ensures that the ratio ^{hi)/(j){hi) is bounded and since (f) is increasing, we 
get 



F211 < c 



E'^(^*)'" 



i=l 



fc=ip=i 



,2-2^ ^ Bn,2-2t C. 



K.i-i n 



Hence, 

n4){hn)F21l = O {(t){hn)Cn) ■ 

Now for the second term F212, again, using the fact that (j) is an increasing function, we get 



(10) 



212 



< c 



E'^(^^)'" 



i=l 



-2 



fc=ip=i 



,2-2£ ^ ^n,2-2£ C,, 



^n,l-^ " 



SO that 

n(t){hn)F2l2 = O {(p{hn)Cn) ■ 

Prom (fTUj) and pT|) we deduce 

n<j){hn)F2l = O {4>{hn)Cn) ■ 

Next, for the second term F22 in ([7]), we have from BiUingsley's inequahty, 

"^ n— 1 n 



(11) 

(12) 



i^22 < C 



< c 



E'^(^^)' 

n 

E'^(^^)' 



-^ 



j; ^A:-^</.(/ifc+p)-V(^)" 

fe=c„+l p=l 



,i=l 



-2 



1-p ra „ 

——rl^HK) <t>{hp) ' <c 
P P=i 



i-p 



Bl i_£ n(t){hnf 



Therefore 



nct>{K)F22 = O ^ 



_2 

If we choose c„ = [</)(/i„) f J, we deduce from (fT2]) and ([13]) that 



(13) 



p-2 



n(j){hn)F2 = O {(j){hn) p ) = 0(1) as long as p > 2, 

and the first part of Lemma [2] follows. Now, as in the proof of the first part of Lemma [21 the variance 
term of ipW is decomposed as follows 



Var(v.M(x)) 



E^(^^)'~ 



i=l 






h+I'. 



i=l 



where here Aij denotes for any integers i and j as follows Aij = F{hi)^^F{hj)^^Cav {YiKi{x), YjKj{x)) ■ 
The study of the term Ii is treated in the same manner as in the independent case (see Amiri et al. 
[2] for more details) which gives 

nc/>(M/i = fe^^M2 [1 + o(l)] . 
^[i-i] /i(x) 

6 



Now, for the second term /2, we always consider a sequence of real numbers c„ which goes to oo as 

n — 7- oo and we write 



2 ( Z/fc=i z2p=i \^k+p,p\ + z2k=c„+i z2p=i \^k+p,p\ I 



[T.unhi)'-'x 



21 ^-'22- 



(14) 



The term I22 is treated exactly as -F22 in the proof of the first part of this Lemma previously, by 
substituting the Billingsley lemma with the Davydov lemma. Then, setting 6„ = (^Inn)^''', using ([2]) 
and with the help of (H5), we get 



I22 < c(ln n) 



2/^^ 



Y.^{h,)'' 



i=l 



.l-p/2 



< c(ln n 



.2/n^n^^ 1_ 
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.l-p/2 
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Therefore, 



\-i 



(15) 



nHK)l22 = O i^{lnnf/>'ci-P/\(P{hn)y 
For the second term I21, observe that for any integers i and j, 

\Cov{YiK,{x),YjK,{x))\ < \E [Y,YjKi{x)Kj{x)]\ + |E [YiK,{x)]\ |E [YjK,{x)]\ . 
Now, from assumptions (HI) and (H2) and conditioning on X, one have 

E[Y,Ki{x)] = MiF{h,)[r{x)+j^]<ccl){hi), 

where 7^ goes to zero as i — )• 00. Using Cauchy- Schwartz' inequality, choosing 6„ = (Inn)^''^ and ([2]), 
we get 

\Cov{Y,K,{x),YjK,{x))\ < E'/^Y^Y^']E'/'[k!{x)K]{x)] 

+ \E[Y,K,{x)]\\^[Y,K,ix)]\ 
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(lnn)2/^V'(/i.)'/V(/i,)'/' + Hhi)Hhj] 



The rest of the proof for I21 is the same as the one for F21 which implies that I21 < c^ [(Inn)^'^ + l] 
Hence, 



ncP{hn)l2i = O fc„0(/i„)(lnn)2/^ 



(16) 



and the result of the second part of Lemma [2] follows from (jlSp and ()16p with the choice c„ = [</>„ '' J . 
Next, to treat the last part of LemmaO it suffices to decompose the term n0(/i„)Cov ( (/>« (x), fn (x) ) 
by the principal and covariance terms and use the same procedure as in the proof of the second part 
of Lemma [2j D 
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In n 
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{^L'kx) - rix)fi!\x) - E [^L'kx) - r{x)f!!\x)] } 

where (p is defined in ([5]). Under assumptions (H1)-(H6), we have 

ISI N <2[l + Vt{x)] as., 

n— ^-oo 

where Vi is defined in ^. 



Proof. Set Wn,^, 



i^,(x)[y.l{|y,|<fe„}-r(x)] 



where Zn^i = Wn,i — 'EWn,i- To prove Lemma [3l we 



use the blocks decomposition technique. Let pn and g„ be some sequences of real numbers defined by 
Pn = Lpo InnJ with po > and g„ = [i^\ ■ Set 

(?n (?n -. n 



i=i 



i=i 



k=2p„qn + l 



with K(j) = ^ E ^n,fc, j = 1, • • • , 2g„. Then we have N = S'^ + S^' + S'^'. Observe that 

k=(j-l)pn + l 

the third term S"' is negligible so that, to prove the strong consistency of N, it suffices to check the 
almost sure convergence for 5(j + S". For any e > 0, 

p (|5; + s':\ > e) < p {\s'j > I) + p {k\ > I 

We just treat 5*^, the term S'^ being similar. Since K is bounded and (j) is non decreasing, we get for n 
large enough |ln(j)| < f (y)B n6(h ) ■ Using Rio's [20] coupling lemma, the random variables V„(j) 
can be approximated by independent and identically distributed random variables V*{j) such that 

E |K(2i - 1) - v:{2j - 1)1 < 4||E:|Up„6 

Since Pn^n ^ 'i-j it follows that 

Qn All Tk^W h 
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a{pn). 



Therefore, for any e, k > 0, Markov's inequality leads to 
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Next setting e„ = e 
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with e > and A„ = ^Jn(l){hn) Inn, we have from (H4)(iii), |AnV^(j)| < 
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0, therefore for n large enough, |A„V^*(j)| < i. It follows that 
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From Markov's inequality, we get 
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, we will assume for the moment 
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where 14 is defined in ([3]). It follows from (fTSj) and (|T9|1 that, for n large enough, 

9n 



\lY,^V:\2j - 1) < y,(x)lnn[l + 0(1)] . 



Therefore 
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Now, combining (fT7|l and (pO|) . we get 
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Next, with the choice of 6„ = {5\nnY'^, the conclusion follows from the application of the Borel- 
Cantelli's lemma whenever po> - and e > 2(1 + k) [1 + V£(x)], which implies that 



lim 

n— >oo 



n(t>{K) 
In n 



1/2 



N<2il + K)[l + Veix)] a.s.. 



for all positive k and Lemma [3] follows. To complete the proof, let us prove (fTHjl and (fT9]l . We can 
write 

"^^^"^^ VVar(Z„fc) = ^Li^( ^fc)~^^Var(i^.(x) [^a{iy,|<M " ^(x)]) 



n 



\l-2£ 



,=1 mx)Bi^,^,n^{K) 

Following the same lines of the proof of Lemma 5 in Amiri et al. p], one can prove that 

n 

k=l 
therefore (jlSp follows. Next, about the covariance term in (J19p . for any integers i ^ j, let 

A,, = F{h,y'F{h,y' |Cov (k,(x) [ya{|y,|<6„} -r(x)] ,i^,(x) [lS-l{|y,|<6„} -r(x)]) 



Then, we have 



n^ 



2_^ Cov{Zn,k,Zn,k') < 






fcT^fc' 



/i(x)^li-,n2</.(/i„ 



\2-2^ 



J1 + J2 



Using BilHngsley's inequahty, one can prove that n(j){hn)J2 = O (h'^Cn '^(j){hn) ^ ) . Next, smce 



|^fc+P,p| < (6n + \r{x)\f [IE {Kk+p{x)Kp{x)) + E{Kk+p{x)MKp{x))] 

< c{bn + \rix)\f [ip{hk+p)'4){hp) + (t){hk+p)(t){hp)] <j){hk+py'^(t){hpY^ . 

Therefore, as in the proof of the first part of Lemma [21 we get 

n(j){hn)Jl = 0{bl(j){hn)Cn), 

which together with the choice c„ = [</>(/i„) p \ imply ()19p as long as p > 2. D 

4.2 Proofs of the main results 
4.2.1 Proof of Theorem [1] 

Let us consider the decomposition (jl|). For the residual term '^" [q '^" , following the same lines of 

Jn [X) 

proof in Amiri et al. [2j by replacing ^rj^\ by ^PiJ^ \ , one can show that 



n(t){hn 
Inn 



1/2 



v^L^i(x)-^M(x) 



a.s, when n — )• 00. 



(22) 



For the principal term in ([J]), we can write 

^l\x)-r{x)f]^\x) = {^l\x)-r{x)f^\x)-^[p'i\x)-r{x)f]^\x)\} 

+ {E[^M(x)-r(x)/M(x)]}. (23) 

Noting that, from Lemma 3 in Amiri et al. p], we have E ( /« (x) ) = -^i[l + o(l)] and it can be 
shown as the same lines of the proof of Lemma [3] that 



/^l(x)-Ktel(x))=0 



/ Inn 
n<j){hn) 



Therefore, Theorem [T] follows from the combination of Lemmas [T] and O since from Lemma [TJ if 



lim n/i^ = 0, then lim 

n— ^+00 n— ^oo _ 



n4>{hn) 
In n 



'^'{E[<^M(x)-r(x)/^l(x)]}=0. 



4.3 Proof of Theorem [1 

The mean square error of Vn can be decomposed as follow: 

r2(x)Var(/|'](x)) 2r(x)Cov (^^'^(x), /^'^(x) 



E 



[rl^Kx)-rix) 



mfl^\x)) 



mfl^kx)) 



Var(^^M(x)) E^l(x) - r(x)E/M(x) 



+ 



E2(/M(;^)) E2/r(x) 

+o(/i2) + o(l/(n</.(/i„))). 



Theorem [2] follows from Lemmas [H - [21 



D 
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